Comparative analysis of computer-vision and BLE technology based indoor navigation systems for people with visual impairments

Background: Considerable number of indoor navigation systems has been proposed to augment people with visual impairments (VI) about their surroundings. These systems leverage several technologies, such as computer-vision, Bluetooth low energy (BLE), and other techniques to estimate the position of a user in indoor areas. Computer-vision based systems use several techniques including matching pictures, classifying captured images, recognizing visual objects or visual markers. BLE based system utilizes BLE beacons attached in the indoor areas as the source of the radio frequency signal to localize the position of the user. Methods: In this paper, we examine the performance and usability of two computer-vision based systems and BLE-based system. The first system is computer-vision based system, called CamNav that uses a trained deep learning model to recognize locations, and the second system, called QRNav, that utilizes visual markers (QR codes) to determine locations. A field test with 10 blindfolded users has been conducted while using the three navigation systems. Results: The obtained results from navigation experiment and feedback from blindfolded users show that QRNav and CamNav system is more efficient than BLE based system in terms of accuracy and usability. The error occurred in BLE based application is more than 30% compared to computer vision based systems including CamNav and QRNav. Conclusions: The developed navigation systems are able to provide reliable assistance for the participants during real time experiments. Some of the participants took minimal external assistance while moving through the junctions in the corridor areas. Computer vision technology demonstrated its superiority over BLE technology in assistive systems for people with visual impairments. - 2019 The Author(s).Scopu

A Scene-to-Speech Mobile based Application: Multiple Trained Models Approach

The concept of Scene-to-Speech (STS) is to recognize elements in a captured image or a video clip to speak loudly an informative textual content that describes the scene. The contemporary progression in convolution neural network (CNN) allows us to attain object recognition procedures, in real-time, on mobile handled devices. Considerable number of applications has been developed to perform object recognition in scenes and say loudly their relevant descriptive messages. However, the employment of multiple trained deep learning (DL) models is not fully supported. In our previous work, a mobile application that can capture images and can recognize the objects contained in them was developed. It constructs descriptive sentences and speak them in Arabic and English languages. The notion of employing multi-trained DL models was used but no experimentation was conducted. In this article, we extend our previous work to perform required assessments while using multiple trained DL models. The main aim is to show that the deployment of multiple models approach can reduce the complexity of having one large compound model, and can enhance the prediction time. For this reason, we examine the prediction accuracy for single DL model-based recognition and multiple DL model-based recognition scenarios. The assessments results showed significant improvement in the prediction accuracy and in the prediction time. In the other hand, from the end user aspect, the application is designed primarily for visually impaired people to assist them in understanding their surroundings. In this context, we conduct a usability study to evaluate the usability of the proposed application with normal people and with visually impaired people. In fact, participants showed large interest in using the mobile application daily.ACKNOWLEDGMENT This publication was supported by Qatar University Collaborative High Impact Grant QUHI-CENG-18/19-1. The findings achieved herein are solely the responsibility of the authors. The contents of this publication are solely the responsibility of the authors and do not necessarily represent the official views of the Qatar University.Scopu

Smartphone-based food recognition system using multiple deep CNN models

People with blindness or low vision utilize mobile assistive tools for various applications such as object recognition, text recognition, etc. Most of the available applications are focused on recognizing generic objects. And they have not addressed the recognition of food dishes and fruit varieties. In this paper, we propose a smartphone-based system for recognizing the food dishes as well as fruits for children with visual impairments. The Smartphone application utilizes a trained deep CNN model for recognizing the food item from the real-time images. Furthermore, we develop a new deep convolutional neural network (CNN) model for food recognition using the fusion of two CNN architectures. The new deep CNN model is developed using the ensemble learning approach. The deep CNN food recognition model is trained on a customized food recognition dataset.The customized food recognition dataset consists of 29 varieties of food dishes and fruits. Moreover, we analyze the performance of multiple state of art deep CNN models for food recognition using the transfer learning approach. The ensemble model performed better than state of art CNN models and achieved a food recognition accuracy of 95.55 % in the customized food dataset. In addition to that, the proposed deep CNN model is evaluated in two publicly available food datasets to display its efficacy for food recognition tasks.This publication was made possible by Qatar University collaborative grant number QUCG-CED-20/21-2 from the Qatar University. The findings achieved herein are solely the responsibility of the author.Scopu

CamNav: a computer-vision indoor navigation system

We present CamNav, a vision-based navigation system that provides users with indoor navigation services. CamNav captures images in real time while the user is walking to recognize their current location. It does not require any installation of indoor localization devices. In this paper, we describe the techniques of our system that improve the recognition accuracy of an existing system that uses oriented FAST and rotated BRIEF (ORB) as part of its location-matching procedure. We employ multiscale local binary pattern (MSLBP) features to recognize places. We implement CamNav and conduct required experiments to compare the obtained accuracy when using ORB, the scale-invariant feature transform (SIFT), MSLBP features, and the combination of both ORB and SIFT features with MSLBP. A dataset composed of 42 classes was constructed for assessment. Each class contains 100 pictures designed for training one location and 24 pictures dedicated for testing. The evaluation results demonstrate that the place recognition accuracy while using MSLBP features is better than the accuracy when using SIFT features. The accuracy when using SIFT, MSLBP, and ORB features is 88.19%, 91.27%, and 96.33%, respectively. The overall accuracy of recognizing places increased to 93.55% and 97.52% after integrating MSLBP with SIFT with ORB, respectively.This publication was supported by Qatar University Collaborative High Impact Grant QUHI-CENG-18/19-1. The findings achieved herein are solely the responsibility of the authors. The contents of this publication are solely the responsibility of the authors and do not necessarily represent the official views of the Qatar University.Scopu

Indoor positioning and wayfinding systems: a survey

Navigation systems help users access unfamiliar environments. Current technological advancements enable users to encapsulate these systems in handheld devices, which effectively increases the popularity of navigation systems and the number of users. In indoor environments, lack of Global Positioning System (GPS) signals and line of sight with orbiting satellites makes navigation more challenging compared to outdoor environments. Radio frequency (RF) signals, computer vision, and sensor-based solutions are more suitable for tracking the users in indoor environments. This article provides a comprehensive summary of evolution in indoor navigation and indoor positioning technologies. In particular, the paper reviews different computer vision-based indoor navigation and positioning systems along with indoor scene recognition methods that can aid the indoor navigation. Navigation and positioning systems that utilize pedestrian dead reckoning (PDR) methods and various communication technologies, such as Wi-Fi, Radio Frequency Identification (RFID) visible light, Bluetooth and ultra-wide band (UWB), are detailed as well. Moreover, this article investigates and contrasts the different navigation systems in each category. Various evaluation criteria for indoor navigation systems are proposed in this work. The article concludes with a brief insight into future directions in indoor positioning and navigation systems.This publication was supported by a Qatar University Collaborative High Impact Grant QUHI-CENG-18/19-1. The findings achieved herein are solely the responsibility of the authors. The contents of this publication are solely the responsibility of the authors and do not necessarily represent the official views of Qatar University. AcknowledgementsScopu